Skip to content

feat(prose-tests): the mid-flow substitution, and a world only prose can describe#555

Open
leeovery wants to merge 1 commit into
prose-tests/asserter-knows-stubsfrom
prose-tests/discussion-conversation-stub
Open

feat(prose-tests): the mid-flow substitution, and a world only prose can describe#555
leeovery wants to merge 1 commit into
prose-tests/asserter-knows-stubsfrom
prose-tests/discussion-conversation-stub

Conversation

@leeovery

@leeovery leeovery commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • Exercises the conversation stub — designed at length, never used. Writing the case proved the mechanism needed no change: a stub is named content, the case owns the trigger, and "once the discussion file has been created from the template, the map seeded, and the initialisation commit made" is as observable a moment as an engine dispatch. The stub writes the worked discussion and settles the map, then the walk resumes.
  • Adds a third shape of end state, which the case forced. A discussion's subtopics are derived by the model from available context — "Derive initial subtopics from whatever context is available" — so their names and number differ run to run and no recipe can pin the resulting world. Such cases now set "world": "claims" in case.json: the delta is taken against the fixture, expected non-empty, and judged against the case's stated claims.
  • It is deliberately the weakest of the three shapes and documented as a last resort — validation refuses claims alongside an assertion-state.cjs, because a world a recipe can build must be built.

The case itself walks initialisation for real: resume detection on a missing file, topic start before any map command (the map requires the item to exist), the file from the template with Triage seeded (none), subtopics recorded pending, the commit — then substitutes the session and stops at the review boundary without dispatching anything.

Test plan

  • npm test green: 1699 tests, 0 fail. Corpus: 13 cases, 2 stubs, valid.
  • Fixture snapshot built and byte-deterministic.
  • Not yet walked by an agent — that's the next run, and it's the first real test of the mid-flow substitution.

🤖 Generated with Claude Code

Stack

  1. docs(design): prose-tests programme design log #544
  2. feat(prose-tests): the framework — cases, worlds, runner, skill #545
  3. test(prose): feature happy-path corpus — five worlds, seven cases #546
  4. test(prose): bugfix corpus — the investigation-centric surfaces #548
  5. test: retry recursive teardown removals — kill a class of phantom failures #549
  6. fix(entry-skills): close the handoff fences — six files render their arms wrong #550
  7. docs: a contributing page for working on the system #551
  8. fix(entry-skills): every handoff arm says to invoke the skill #552
  9. fix(implementation): environment setup belongs to the setup reference alone #553
  10. fix(prose-tests): the asserter is told which substitutions were armed #554
  11. feat(prose-tests): the mid-flow substitution, and a world only prose can describe #555 👈 current
  12. test(prose): claims assert consequences, not what was displayed #556
  13. feat(prose-tests): record everything the agents do, results included #557
  14. fix(discussion-entry): the handoff reports the source it actually had #558
  15. fix(prose-tests): the stop hook records, and names the model that walked #559
  16. fix(prose-tests): command output was never actually recorded #560
  17. feat(prose-tests): judge the walk as told, not the summary returned #561
  18. feat(prose-tests): decide in code what an agent should not be deciding #562
  19. test(prose): a case starts where a session starts #563
  20. feat(prose-tests): walk on Sonnet, judge on Opus, escalate a failure #564
  21. test(prose): give the eight read-only cases something that can fail #565
  22. test(prose): only walks that can be observed, and checks that survive the trip #566
  23. fix(prose-tests): the verdict names only the model the record names #567
  24. test(prose): discovery, walked to the point where work first exists #568
  25. fix(prose-tests): the asserter judges which of prose or walker was at fault #569
  26. docs(conventions): a step whose reference routes every exit still signposts #570
  27. test(prose): discovery's epic arm, to the same durability boundary #571
  28. fix(prose-tests): keep the walk's closing turn, and name the one live difference #572
  29. fix(prose-tests): prescribe the cross-check, drop a check that cannot fire, lint the scope #573
  30. test(prose): declare the prose each case actually walks #574
  31. fix(investigation): the symptom interview never ran for work shaped in discovery #575
  32. feat(prose-tests): conduct — playing a person where a script cannot reach #576
  33. fix(prose-tests): what a write put in a file is evidence, not incident #577
  34. fix(investigation): the interview starts from the carrier, not over it #578
  35. test(prose): quick-fix scoping, the whole definition stage in one walk #579
  36. fix(prose-tests): the world log is the walker's record #580
  37. fix(prose-tests): a confirmed failure reports both runs' checks #581
  38. fix(prose-tests): each prescribed command runs as written #582
  39. fix(scoping): context gathering reads the whole carrier, asks only the gaps #583

@leeovery
leeovery force-pushed the prose-tests/asserter-knows-stubs branch from db46617 to e81714b Compare July 26, 2026 13:37
@leeovery
leeovery force-pushed the prose-tests/discussion-conversation-stub branch from c9f9f1a to f83df8e Compare July 26, 2026 13:37
@leeovery
leeovery force-pushed the prose-tests/asserter-knows-stubs branch from e81714b to f4cfb5a Compare July 26, 2026 13:43
@leeovery
leeovery force-pushed the prose-tests/discussion-conversation-stub branch from f83df8e to 9d3d1b3 Compare July 26, 2026 13:43
This was referenced Jul 26, 2026
…can describe

The conversation stub was designed at length and never exercised. Writing
the case proved the mechanism needed no change — a stub is named content,
the case owns the trigger, and "once the file is created from the
template, the map seeded, and the initialisation commit made" is as
observable a moment as an engine dispatch. What the case did need was a
third shape of end state.

A discussion's subtopics are derived by the model from whatever context
it has, so their names and number differ run to run and no recipe can
pin the resulting world. Cases like that now set `world: "claims"`: the
delta is taken against the fixture, expected to be non-empty, and judged
against the case's stated claims. It is the weakest of the three shapes
and reached for last — a world a recipe can build must be built, and
validation refuses claims mode alongside an assertion state.

The case walks discussion initialisation for real — resume detection,
topic start before any map command, the file from the template, subtopics
recorded pending, the commit — then substitutes the session it cannot
simulate and resumes to the review boundary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leeovery
leeovery force-pushed the prose-tests/asserter-knows-stubs branch from f4cfb5a to fc96a56 Compare July 27, 2026 14:32
@leeovery
leeovery force-pushed the prose-tests/discussion-conversation-stub branch from 9d3d1b3 to 644f09b Compare July 27, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant